Multiple coordinated detectors for examination and ranging

ABSTRACT

This invention focuses specifically on the use of epipolar lines and the use of matrix transformations to coordinate cameras. This invention organizes cameras in a manner which is intuitive and effective in perceiving perspectives which are not normally possible; to calculate range precisely; to allow redundancy; to corroborate feature recognition; and to allow perspectives from angles from which no cameras exist. By enabling remote scene reconstruction with a limited set of images, transmission bandwidth is greatly conserved.

BACKGROUND OF THE INVENTION

The ability to coordinate detectors accurately, and the increasing powerof computers, allows for the accumulation and organization of image datain a manner not formerly possible, at very high speeds and at arelatively low cost. The many applications include quality control inproduction lines, examination of internal organs, facial recognition andmissile tracking.

To focus on the last problem, or opportunity, we know that the recentavailability of ultra-high-speed processors allows the computation ofhighly complex data in speeds approaching real-time. With fast imagerecognition algorithms and high-speed software, 3D ranging can be donein milliseconds. This allows equally fast (and automated) response toincoming aircraft or missiles threatening a military asset—a tank, aradar station, a navy ship—all the while the missiles being unaware thatthey are being tracked and therefore less capable of taking evasive orjamming action.

A missile approaching at 600 mph will take six seconds to cover a mile.Its identity and vectors of range, bearing, velocity, etc. must begrasped instantly for evasive or defensive action to be taken.

Ranging relates to perception in three dimensions in that an objectneeds to be seen from two or more points of view in order to calculateits range and properly determine its character.

As the Navy puts it: “Three-dimensional imaging technology, using imagedata collected from multiple offset cameras, may be able to passivelyprovide the automated ranging capabilities to the war fighter that werepreviously only available through active systems that risked thepossibility of counter-detection in their use.”

This invention focuses specifically on the use of epipolar lines and theuse of matrix transformations to coordinate detectors: to organize themin a manner which is intuitive and effective in perceiving perspectiveswhich are not otherwise possible; to make detectors effective over longand short ranges; to calculate range precisely; to allow redundancy; andeven to allow perspectives from angles from which no detectors exist.Until recently the possibility of doing all these things virtuallysimultaneously did not even exist.

DESCRIPTION OF THE RELATED ART

The related art of identifying an object distance has two components (i)Passive ranging, as we do in ambient light with our eyes; and (ii)Active ranging, which involves illuminating an object with searchlights,flares, lasers, radar, etc. so that we can detect its distance, shape,velocity, etc. in the visible, infra-red, microwave, millimeter waveregions. Active ranging also includes sonar.

(i) Passive Ranging:

Single cameras held still, no matter how large and how high theirresolution, can do little to create a perception of depth. Binocularsmake distant scenes appear flat, and because of the small separationbetween their optics, cannot easily calculate range. Depth cannot easilybe created using phase or wavelength differences or by other known meanswith two adjacent and essentially parallel light paths or beams, else wecould create 3D images of stars with relatively closely spaced detectorslike a pair of eyes, separated by only 65 mm. Though some believe thatanimals can achieve ranging by heterodyning wavelength differenceswithin single eyes.

Thermal (infra-red) imaging and night-vision devices suffer similarissues. They take second place to visible estimation, since theirresolution is not as good. For thermal imaging the wavelengths arelonger and in night-vision the images tend to be fuzzy andindeterminate.

Stadimeters have been used by the armed forces of a number of countriesfor passive ranging since the 1800s. They rely on two well-separatedincoming light beams, with prisms or mirrors to combine them on ascreen, whose images can be seen simultaneously and made to overlap bymanual dialing. On the dial is a scale which shows larger and largernumbers rapidly converging with distance.

Motion parallax, where a set of eyes move from one position to another,has been used since time immemorial for measuring distance, and enablespeople and animals to estimate distance by looking at an object frommore than one point of view. It was certainly used by the Greeks inconstructing their temples, and by Eratosthenes in Egypt in 200 BC incalculating the diameter of the earth and the distance to the sun.

Another feat of passively estimating distance was The GreatTrigonometrical Survey of India begun in 1802. In forty years, throughimmense heart and labor, a Great Arc of triangles reached the Himalayasfrom a base-point on the sea. Using broad triangles many efforts weremade (on clear days) to establish heights of certain distant peaks inthe Himalayas. From 1847 to 1849 one elusive cluster of peaks wascontinuously re-examined and recalculated for height. In 1856 it wasmade public: at 8,850 meters or 19,002 feet (then), Mt. Everest wasclaimed as the world's tallest mountain.

Both Greeks and British understood the value of broad baselines andaccurate measurements for passive ranging. Triangulation still remains apowerful tool for passive measurements of objects and distances.

Triangulation also helps with sound—our ears can determine the directionof sound. Armies (such as North Vietnam's) used multiple sonar dishes tocalculate the range of US bombers.

(ii) Active Ranging:

LIDAR and RADAR have been used for many years for ranging, detection and(with associated computers) analysis of objects. However, as the Navysays: “active ranging, conducted via lasers or radar emission, andactive transmission is undesirable in many circumstances. An increase inthe fidelity of passive ranging and automation of the process of passiveranging will reduce the requirement for active transmission.”

Although lasers have considerable range, the scattered light does not:it reduces as the square of the distance, limiting the range of lasersto a few hundred yards. And while radar and millimeter waves have theability to penetrate clouds and fog, in the visible spectrum theresolution of their images is orders of magnitude less than with oureyes.

Bats and dolphins use sonar for estimating range, the first in airbeyond the audible range, the second at low audio wavelengths forpenetrating seawater. The time of flight of sound-waves can give them anaccurate estimation of distance. But it is not passive.

In principle night-vision devices can be used for passive ranging but(as noted) may not be as effective as the latest in low light levelimaging devices because of poor image quality. In general night-visiondevices are expensive and require high voltages to run.

SUMMARY OF THE INVENTION

This invention focuses specifically on the use of epipolar lines and theuse of matrix transformations to coordinate detectors. It organizes themin a manner which is intuitive and effective in perceiving perspectiveswhich are not otherwise possible; to make detectors effective over longand short ranges; to calculate range precisely; to allow redundancy; andeven to allow perspectives from angles from which no detectors exist.Through this we try to overcome certain limitations of existing imagingtechnology.

With modern computers we can recognize objects with templates andaccurately triangulate their distances (with multiple baselines) inmilliseconds.

Diagrams are given which show geometrically how this is achieved. Somenotes are added on the mathematics necessary for comparing multipleimages, and for the transforms needed for comparing one format toanother.

What we will show in this invention that this is an efficient andexact—and therefore fast—way of creating wholly real new images fromadjacent real images, in other words to create imaginary cameras whichwe can place at will.

Because whole scenes can be re-constructed remotely in virtuallyreal-time by anyone with a powerful computer, there can be an enormoussaving of bandwidth in transmission.

The techniques of this invention are applicable in many areas. Forexample they are useful in internal medicine for haptic feedback, inrailways for the inspection of ties, in production lines for theinspection of finished parts, at security facilities for facialrecognition, and especially in forensics. With increasingly fastcomputers capable of handling vast amounts of data, imaging operationsin all these areas can be done in milliseconds.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 shows a basic geometrical layout of two detectors for seeing orsensing an object 200, with the parameters of width and length of theobject and the corresponding images shown for computation. Thedetectors, or cameras, are described as an image plane 10 with lens 1,and an adjacent and coplanar image plane 120 with lens 12. The notationis typical for triangulation.

FIG. 2 shows a graphs 901 and 902 the results of using the parametersdescribed in FIG. 1 for computations of length and width in the object200.

FIG. 3 shows a projection of the object 200 into 3D space, with theimage planes 10 and 120 convergent towards object 200, showing images ofthe object on both planes, and with a baseline 304 connecting the cameracenters 1 and 12. (Noted here is the theoretical convention inprojection that the lenses 1 and 12 become now the camera centers 1 and12 with the image planes 10 and 120 inverted towards the object 200,simplifying exegesis.) Where the baseline 304 intersects the imageplanes 10 and 120 are the epipoles e and e′, important in what follows.

FIG. 4 shows that by projecting the epipoles e and e′ to infinity wemake the image planes 10 and 120 coplanar, and the lines 301, 302 and303 joining similar salient features in the two images, such as imagecorners 15 and 121, are made parallel to the baselines.

In FIG. 5 this concept is extended to multiple camera centers 1 . . . 12(described later as “nodes”) which for the sake of illustration (andsimplicity) we here create as a symmetrical dodecagon 1000. It will beshown later that it can be any shape at all, both planar and non-planar,and that we can derive special relationships from the parallelism oflines on the epipolar pencils.

In FIG. 6 is shown how “missing” cameras affect imaging. We haveeliminated cameras at alternating nodes (such as 1, 3 and 5) leaving ahexagon of even numbered nodes (such as 2, 4, 6). Note here that animage 20 constructed with a reduced camera set (6) for node 2 is exactlythe same as it is for a full camera set (12). This is to indicate howworking with a “skeleton” can be effective. Later we will show thatskeletons can be of any shape.

FIG. 7 shows how an imaginary camera 13, located randomly, can be madeto perform as a real camera within a skeleton (a polygon) formed byother camera centers.

FIG. 8 shows the parameters of a polar coordinate system. This is toshow how polar geometry can relate to outlying cameras which fall offour usual definition of coplanar.

In FIG. 9 illustrates how a point X can project onto two differentcoordinate systems, but whose images can still be related throughvarious coordinate transformations

In FIG. 10 shows a visual range horopter 501 of all the coordinateddetectors 500 in a detector bundle, as will be described below.

FIG. 11 shows a possible way of processing the data gathered by cameras1, 2, 3 . . . n. For redundancy (in case any lose function) a parallelconfiguration of cameras is preferred. When an object (such as anaircraft) is recognized by the computer, a select few cameras as in 1,2, 3 . . . etc. can be chosen to track it, depending on the object'srange.

In the discussion which follows the imaging devices will be referred toas cameras, detectors, or nodes, as apt in context. “Detectors” is amore general term, referring to any device capable of perceiving imageswithin the electromagnetic spectrum or sonar range.

DETAILED DESCRIPTION OF THE DRAWINGS

This invention, with its further advantages described below, may be bestunderstood by relating the descriptions below to the drawings appended,wherein like reference numerals identify like elements, and where:

FIG. 1 illustrates how the geometrical parameters of an object,including depth, can be measured using two imaging devices. A firstimaging device 401 is defined by a lens 1 and an image plane 10. Asecond imaging device 412 is defined by a lens 12 and an image plane120. In the configuration shown the image planes 10 and 120 are coplanarin the x-y plane and lie centered on the x-axis.

In FIG. 1 the imaging devices 401 and 412 are looking at an object, orfeature, 200 in 3D space from two different perspectives. In ourillustration the feature 200 is shown as a flat object such as a kite(for flying) or a sail (on a boat). The corners 201, 202 and 203 offeature 200 are imaged onto image plane 10 through lens 1 as points 15,16 and 17. The corners 201, 202 and 203 of feature 200 are imaged ontoimage plane 120 through lens 12 as points 121, 122 and 123.

In FIG. 1, given f as the focal length of the imaging devices, orcameras, and +h and −h as the distances of the lenses from the z-axis,we can compute the pixel offsets q₁ and q₂ corresponding to thedistances d₁ and d₂ to the feature corners 201 and 202, which in thisscenario happen to lie along the z-axis. Geometricallyq ₁ d ₁ =q ₂ d ₂ =hfor simply hf=qdwhere, with the cameras secured and h and f both constant, the variablesq and d describe a hyperbolic curve 901 as shown in FIG. 2.

As an example from this curve 901 suppose that f is 35 mm, h is onemeter, the detector pixels are 2μ, and there is an image offset q on theimage plane of 50 pixels. Thend=35×10⁻³×1/50×2×10⁻⁶=350 metersHere d could exemplify the distance from the lenses 1 and 12 along thez-axis to feature corner 202. Correspondingly if on the image the valueof q were increased by 8 more pixels then d would become 300 meters,making image 200 larger and bringing feature corner 201 50 meters closerto the detectors.

FIG. 1 also shows offsets from the x-z plane of point p (14) in the +ydirection corresponding to a offset l in the −y direction of the featurecorner 203. At a specific distance d along the z-axis this creates alinear relationship 902 as shown in FIG. 2p=fl/d

As a calculation from this line 902 in FIG. 1 suppose that f is 35 mm, dis a thousand meters, the detector pixels are 2μ, and there is a featureoffset of 50 pixels. Thenl=50×2×10⁻⁶×1000/35×10⁻³=30 metersThis shows that the calculation of offsets from the z-axis is linear. Itis a lot more sensitive than the calculation of distance along thez-axis, especially as distances increase.

A series of lenses in the following discussion (such as 1, 2, 3, . . .12, etc.) will be referred to as “camera centers”. It is useful (andconventional) to put the image plane in front of the lenses since,instead of being inverted, the images can be seen to more closelycorrespond to the objects being studied. For reasons which follow thecamera centers will also be abbreviated in the discussion as “nodes”.

In FIG. 3 is shown a projection of the image planes 10 and 120 into 3Dspace where the lenses 1 and 12 become the camera centers and the points210, 202 lie on a plane 415. In this figure the image planes 10 and 120are shown tipped inwards, or convergent, towards the points 201, 202 and203.

In FIG. 4 we project the epipoles e and e′ to infinity, thereby makingthe image planes 10 and 120 in this imaging system coplanar. Thisprojection has the property of making all lines joining similar featuresparallel to the baseline 304, which is the line joining their twocameras centers 1 and 12. For example the corner 201 of object 200 willproject onto detectors 10 and 120 as points 15 and 121 (denoted as x andx′) on a line 301. The diagram shows this line parallel to the baseline304.

In FIG. 5 we extend this concept to multiple camera centers (or nodes)1, 2, 3 . . . 10, 11, 12, etc. For the sake of illustration (andsimplicity) we may make this shape a symmetrical dodecagon. It will beshown later that it can be any shape at all, and can be extended tonon-coplanar detectors.

In FIG. 6 we show how “missing” cameras affect imaging. We haveeliminated cameras at alternate nodes 1, 3, 5, 7, 9 and 11—six in all,leaving a hexagon of 2, 4, 6, 8, 10, 12 nodes as the remainder. Byconnecting new baselines such as 324 between nodes 12 and 2, all similarfeatures 21 and 121, 22 and 122, etc. are connected by parallel lines321 and 322, etc. since the epipoles of image planes 20 and 120 (andothers) are all at infinity.

This has the property that no matter how the baselines are connectedbetween any pair of nodes, the images created on those image planes byany set of lines parallel to the baselines are identical. In otherwords, the image formed by the corners 121, 122, 123 on image plane 120permutes identically to itself no matter how that image plane isconnected to other coordinated image planes on the field.

A missing or unusable imaging device make no difference for imaging inremaining real (or imaginary) imaging devices since using epipolar linesthese can be linked seamlessly to alternate imaging devices in thefield.

This property has huge ramifications in terms of redundancy. The numberof apexes which can be connected in pairs in any field of n coordinateddetectors is n (n−1)/2. This means that 12 detectors (as in FIG. 5) canbe connected together 66 different ways. Losing half these detectors bymishap or sabotage (as in FIG. 6), thereby reducing the number to 6,still allows 15 different combinations, which is statisticallysignificant, and in fact adequate, for many purposes.

In FIG. 7 is shown the ability to place artificial cameras 13 and 14 inplaces where none existed before, and have them function like normalcameras. We will cover this later.

Anticipating FIG. 10, we show a field of coordinated detectors 500. Thisfield may contain many detectors—perhaps a thousand or more for missiletracking, aircraft navigation, drones, etc. The possible number ofdifferent pair combinations for 1,000 detectors is n (n−1)/2 whichequals 499,500—nearly half a million. This implies that any selectfew—perhaps a dozen detectors, may be chosen for progressive tracking atone time. The tracking of multiple targets can overlap and proceedsimultaneously. With massively parallel architecture and instantaneousfault detection should any detector fail (or get blown up in wartime)the system has multiple redundant detectors and resilience to spare. Itcan be programmed to continue seamlessly.

Issues remain: (i) The epipoles are prescribed as being at infinity, buthow must the cameras (or detectors) be coordinated? (iii) how does onego beyond infinity with the epipoles if the cameras are neitherconvergent nor coplanar?

We first address issue (i): How do we coordinate cameras? and why ? (i)We coordinate cameras to reduce computation. Part of this can beachieved—though shown later as not strictly necessary—by acquiring asingle model camera from a given manufacturer. (ii) When the cameras arecoplanar and coordinated failure of any particular camera is not anissue; the computer can sense failure in microseconds; massiveredundancy permits object detection to be switched to other selectedcameras.

To get cameras coplanar with their epipoles projected at infinity weneed (progressively) primary, secondary and fine alignment.

For camera pairs we can enumerate certain physical degrees offreedom—focal length, aperture, zoom, x, y and z, and pitch, roll andyaw. All degrees of freedom must then be adjusted together so thatcameras as pairs and en masse match each other as closely as possible.As examples, the pose of the cameras, i.e. their axes, should beparallel; apertures also should be adjusted to give matching lightintensity on the detectors, etc.

Primary. Assuming cameras are facing skywards and connected in parallel(as they should be), they may be trained on a distant object (a staroverhead), and aligned one by one so that their images coincide (asprecisely as possible by eye) on a computer screen nearby. This willmake them parallel but will not fix image size and rotation, whichfollows.

Secondary. A simple recipe for bringing the images from each pair ofcameras into close parallel, rotation and size correspondence can beperformed in Matlab. It depends on accurately choosing (at least two)matching features in distant images. This could be pinpoints such as twowell-separated and well-known stars. The median (estimated) pixelpositions must be delivered to the program below into the two functionsginput2( ) by the user.

The matching algorithms below we use the local coordinates of thedetectors (rather than the global coordinates discussed later for imagemapping). That is, that when our alignments are carried out to asufficient degree point (x_(i), y_(i)), of image plane 10 willcorrespond (almost) exactly to point (x_(i), y_(i)) of image plane 120.

-   -   alignment.m    -   % load input images    -   l1=double(imread(‘left.jpg’));    -   [h1 w1 d1]=size(l1);    -   l2=double(imread(‘right.jpg’));    -   [h2 w2 d2]=size(l2);    -   % show input images and prompt for correspondences    -   figure; subplot(1,2,1); image(l1/255); axis image; hold on;    -   title(‘first input image’);    -   [X1 Y1]=ginput2(2); % get two points from the user    -   subplot(1,2,2); image(l2/255); axis image; hold on;    -   title(‘second input image’);    -   [X2 Y2]=ginput2(2); % get two points from the user    -   % estimate parameter vector (t)    -   Z=[X2′ Y2′; Y2′ −X2′; 1 1 0 0; 0 0 1 1]′;    -   xp=[X1; Y1];    -   t=Z\xp; % solve the linear system    -   a=t(1); %=s cos(alpha)    -   b=t(2); %=s sin(alpha)    -   tx=t(3);    -   ty=t(4);    -   % construct transformation matrix (T)    -   T=[a b tx; −b a ty; 0 0 1];    -   % warp incoming corners to determine the size of the output        image (in to out)    -   cp=T*[1 1 w2 w2; 1 h2 1 h2; 1 1 1 1];    -   Xpr=min([cp(1,:)0]):max([cp(1,:)w1]); % min x:max x    -   Ypr=min([cp(2,:)0]):max([cp(2,:)h1]); % min y:max y    -   [Xp,Yp]=ndgrid(Xpr,Ypr);    -   [wp hp]=size(Xp); %=size(Yp)    -   % do backwards transform (from out to in)    -   X=T\[Xp(:) Yp(:) ones(wp*hp,1)]′; % warp    -   % re-sample pixel values with bilinear interpolation    -   clear Ip;    -   xl=reshape(X(1,:),wp,hp)′;    -   yl=reshape(X(2,:),wp,hp)′;    -   lp(:,:,1)=interp2(l2(:,:,1), xl, yl, ‘*bilinear’); % red    -   lp(:,:,2)=interp2(l2(:,:,2), xl, yl, ‘*bilinear’); % green    -   lp(:,:,3)=interp2(l2(:,:,3), xl, yl, ‘*bilinear’); % blue    -   % offset and copy original image into the warped image    -   offset=−round([min([cp(1,:)0])min([cp(2,:)0])]);    -   lp(1+offset(2):h1+offset(2),1+offset(1):w1+offset(1),:)=double(l1(1:h1,1:w1,:));    -   % show the results    -   figure; image(lp/255); axis image;    -   title(‘aligned images’);

We can write a more general program in Matlab to bring multiple imageswithin a few pixels of alignment, and consequently make multiple imageplanes parallel simultaneously.

Fine alignment. To get accurate alignment in a terrestrial environmentwe must delve into a “feature-based” approach. In general, for featureselection, any of a number of edge detection algorithms can be used,such as: J. Canny, “A Computational Approach to Edge Detection,” IEEETransactions on Pattern Analysis and Machine Intelligence, Vol. PAMI-8,No. 6, 1986, pp. 679-698). We can apply this to features we have alreadychosen, using the local coordinates of image planes 10 and 120.

Using a notation common in imaging (See Richard Szeliski, December2006), we may utilize the minimum of the sum of squares function ESSD(u)for our step-wise correlation of similar features on image planes 10,20, 30 . . . 120, etc.:E SSD(u)=Σ_(i) [I ₁(x _(i) +u)−I ₀(x _(i))]²=Σ_(i)(ei)²Where u=(u, v) is the feature displacement on orthogonal axes (usinglocal coordinates) and ei=I₁(x_(i)+u)−I₀(x_(i)) is the error function orfeature displacement offset within the feature areas (I₀ being thereference feature on image plane 10 and I₁ a similar feature on imageplane 20, etc.)

That is, we reduce all the errors ei to an acceptable minimum, realizingthat because the images are taken from different perspectives, theerrors ei will never be completely zero.

The sum of squared differences function ESSD(u) above can also bewritten as a Fourier transform:F{E SSD(u)}=F{Σ _(i) [I ₁(xi+u)−I ₀(xi)]²}=δ_((f))Σ_(i) [I ₁ ²(xi)+I ₀²(xi)]−2I ₀(f)I ₁*(f)The right-hand expression shows how ESSD(u) can be computed bysubtracting twice the correlation function (the last term) from the sumof the energies of the two images (the first term). We can use thediscrete cosine transform, DCT-2, if we want to correlate larger pixelareas.

For really fine correlation we can use a partial differential equationto compare the image gradients at the light-to-dark edges of our chosenfeatures on image planes 10 and 120. We can treat the least squaresfunction ESSD(u) above as an energy function with a hypotheticaldisplacement ΔuE SSD(u+Δu)=Σ_(i) [I ₁(x _(i) +u+Δu)−I ₀(x _(i))]²=Σ_(i) [J ₁(x _(i)+u)Δu+(ei)]²where the Jacobian J ₁(x _(i) +u)=∇I ₁(x _(i) +u)=(∂I ₁ /∂x,∂I ₁ /∂y)(x_(i) +u)is the image gradient at (x_(i)+u) and ei=I₁(x_(i)+u)−I₀(x_(i)) is theintensity error (as above).

This is a soluble least squares problem in which sub-pixel resolutioncan be achieved when the Jacobians of the profiles of the two features141 and 142 are approximately equalJ ₁(x _(i) +u)≈J ₀(x)since near the correct alignment the appearance of light-to-dark edgesshould be the same.

This alignment of x and y coordinates will bring the two image planes 10and 120 onto almost identical points (x_(i), y_(i)) on their local x-yplanes, differing only by their global offsets +h and −h from thez-axis, as in FIG. 1. This alignment will apply sequentially to allimage planes 10, 20, 30 . . . up to 120 and beyond.

With the cameras aligned and secured in their locations we can estimatewith fair precision variations in the geometry, recognition and motionof distant objects.

Of interest to us is that the distance h from baseline to detectorscould be quite large—say a hundred meters—making the range accurate at35,000 meters. In addition the separation of pixels could be accuratelygauged at ±1 (or less), making the accuracy for 100 meter detectorseparation 50 times greater again, at 1.7 million meters. More then willdepend on camera resolution.

A reflection is prompted about the resolution by an observationsatellite of asteroid trajectories near earth, especially if there is aneffort to divert an asteroid on collision course. Free from gravity andatmospheric turbulence, cameras could be deployed far from thesatellite. From the calculations above, the range of such cameras 1 kmapart would be 170,000 kilometers, giving ten hours warning for anasteroid approaching at 17, 000 km per hour.

A sum of such images and measurements taken with many cameras willprovide a many faceted picture of an object, as may be seen in theshaded areas of FIG. 5. With increasing facets the ability for softwareto recognize the unique character of an object will steadily improve.Bayesian probabilities will adduce certainty to recognition.

We have not mentioned the additional parameters of color, hue,saturation, light, dark etc., in objects as a means of recognition. Thiswill come later.

FIG. 5 shows an array of coplanar detectors 1 to 12 arrangedsymmetrically around an object 200 lying along a central z-axis betweendetectors 1 to 12. For ease of visualization we make the z-axis lookinto the paper. The object 200 in FIG. 5 is the same object 200 as inFIG. 1, where the central corners 201 and 202 lie on the z-axis, and thecorner 203 lies to one side on the y-axis.

In FIG. 5 the construction lines 211 and 212 connect camera center 1with the corners 201, 202 and 203. The line 211 is shown as single aspoints 201 and 202 lie coincident on the z-axis.

Referring back to FIG. 1, we can transfer the same parameters of lengthd and pixel offsets q, etc. to the more general FIG. 5. On image plane10 the pixel distance q₁ from camera center 1 is plotted as point 15.Similarly the pixel distance q₂ is plotted as point 16, and the pixeldistance q₃ and offset p on the y-axis is plotted as point 17. Weproceed in a similar manner for all detectors 1 to 12.

Because we have arranged it so that epipoles of all these detectors areat infinity, the corners of all these plots form in aggregate nesteddodecagons 1000 (baselines), 1001 (points similar to 15), 1002 (pointssimilar to 16) and 1003 (points similar to 17). These lines togetherform nested sets with parallel sides as shown in FIG. 5.

We can observe from FIG. 1 that the object point 202 furthest away fromcamera center 1 creates the smallest pixel shift q₂ on image plane 10,at point 16. Therefore the aggregate of images of point 202 on all imageplanes 10, 20, . . . up to 120, at points 16, 22 . . . up to 122, formthe smallest dodecagon 1002 in FIG. 5.

Similarly from FIG. 1 the object point 201 nearest from camera center 1creates the largest pixel shift q₁ on image plane 10, at point 15.Therefore the aggregate of images of point 201 on all image planes 10,20 . . . up to 120, at points 16, 22 . . . up to 122, form the largerdodecagon 1001 in FIG. 5.

Again in FIG. 1, the object point 203 intermediate from camera center 1creates an intermediate pixel shift q₃, and an additional pixel offset pin the y-direction, on image plane 10, at point 17. Therefore theaggregate of images of point 203 on all image planes 10, 20 . . . up to120, at points 17, 23 . . . up to 123, form an intermediate dodecagon,offset by an amount corresponding to this pixel offset p, as dodecagon1003 in FIG. 5.

We note that the size of these dodecagons obey the hyperbolic curve 901in FIG. 2. While an object feature 202 (the nose of a missile) is faraway from detectors 1 . . . 12 a dodecagon will appear small (asdodecagon 1002). While object feature 201 (the nose the same missile) iscloser to detectors 1 . . . 12 a dodecagon will appear larger (asdodecagon 1001).

We also note the effect of an object feature 203 (the nose of ourapproaching missile) going off course on the y-axis. This is shown asdodecagon 1003, which is now eccentric. The size of this dodecagon willstill obey the hyperbolic curve 901 in FIG. 2. But in eccentricity thedodecagon will obey the linear curve 902 in FIG. 2. The missile (as wehave named object 200), will in fact track as a dodecagon in a mannercombining curves 901 and 902 from FIG. 2.

Because curve 901 in FIG. 2 is hyperbolic, should a missile actually hitthe detectors the size of the tracking dodecagon would become infinite.

The usefulness of this construction—of which we have given just oneexample—may now be made apparent.

The first is visualization. Through multiple views and with n(n−1)/2comparisons, Bayesian probabilities can rapidly help confirm anidentity. This is important, since images can become indistinct, gobehind clouds, etc.

Another is velocity. Suppose point 202 represents the nose of ourmissile coming directly up the z-axis centerline towards the cameras. Atmoment t₁ it could be at 202. At moment t₂ it could be at 201. As priorcalculations show the separation (as resolved by the detectors) could be50 meters. Given an approach velocity of 600 km/hr. the time differencet₂−t₁ would be 3 seconds. At 350 meters away in the calculations above,this would give a reaction time of just 21 seconds.

To gain time we may propose a change in length of the baseline h betweencameras. If h is changed from one meter to ten, the reaction time willbecome 210 seconds—three and a half minutes. If h is changed from onemeter to one hundred, the reaction time will become 2,100seconds—thirty-five minutes. Multiple coordinated high resolutioncameras with broader baselines will allow greater warning time forreaction.

The addition of many viewpoints beyond 2D not only replicates the visionof the human eye to perceive depth from diverse points of view, but addsvaluable information for the inspection of diverse objects. Theseobjects can be instantaneously compared and sorted againstthree-dimensional templates which may reflect the ideal for thatparticular object. With advanced object recognition software, inspectionand ranging can be done at high speed and with great accuracy.

We now address issue (ii): An epipole of a camera imaging on a flatplane must make its z-axis either parallel to, or convergent with, allothers, else its epipole will either be imaginary or beyond infinity.

A solution to this dilemma is to project a curved image field in frontof the lens to simulate a fish-eye lens or a human eyeball. The epipolewill then fall on a spherical surface, real or imagined, surrounding thecamera center.

This solution is shown in FIG. 9, where a line 406 joining cameracenters 401 and 402 has epipoles e′ passing through a flat plane 410 ande passing through a sphere 412. The pose of this camera with center 402is on the z′-axis. The spherical configuration here makes the poseimmaterial, since within a sphere there is always one radius which willfall parallel to the z-axes of all other detectors. The sphere can bepartial or otherwise, so long as the shape in general is curved. Havingz-axes parallel for all cameras is not only necessary but desirable forsimplifying calculations.

In FIG. 9 we can choose a z-axis (north, polar) 407 as corresponding tothe pose of all other cameras. Using FIG. 8, we convert spherical(polar) coordinates of point 404 (ρ, φ, θ) into (x, y, z) coordinates.This makes images consistent with those from all other cameras. Our newcoordinates are:

-   -   (x, y, z)=ρ(sin φ cos θ, sin φ sin θ, cos φ)        where ρ (rho) is the radius of sphere 412. ρ may also be used as        a scaling factor.

In FIG. 9 any image on a sphere will have barrel distortion and thismust be corrected from image 404 so that it will correspond point bypoint to flat image 403. The following factors will make correspondenceeasier by virtually eliminating this distortion, producing newcoordinates x^(a) and y^(a):x ^(a) =x(1+k ₁ r ² +k ₂ r ⁴)y ^(a) =y(1+k ₁ r ² +k ₂ r ⁴)where k₁ and k₂ are radial distortion parameters and r²=x²+y². r is avariable radius diminishing according to its distance up the z-axis asin FIG. 8. Higher order parameters (e.g. k₃ r⁶) may be necessary foreven more matching towards the poles.

For both Cartesian and Polar coordinate systems we need a way to relateimages between planes 10, 20, 30, . . . n, between spheres such as 412,and between the two imaging systems. We need to compare a matrix Mdescribing an array of pixels x₁ . . . x_(n), y₁ . . . y_(n) to a matrixM′ describing a similar array of pixels x₁′ . . . x_(n)′, y₁′ . . .y_(n)′. In other words we need a transformation matrix T where M′=M.T

Using homogeneous coordinates relating images on flat surfaces thisappears as:

$\begin{pmatrix}x^{\prime} \\y^{\prime} \\1\end{pmatrix} = {\begin{pmatrix}x & y & 1 & 0 & 0 \\y & {- x} & 0 & 1 & 0 \\0 & 0 & 0 & 0 & 1\end{pmatrix} \cdot \begin{pmatrix}{s\;\cos\;\alpha} \\{s\;\sin\;\alpha} \\t_{x} \\t_{y} \\1\end{pmatrix}}$where s is a scaling factor to exactly match image sizes, t_(x) andt_(y) the pixel displacement for precisely corresponding images on theirlocal image planes, and x′ and y′ are the local pixel arrays for thenewly coordinated images. α is the angle of rotation, programmed inradians, normally zero if detectors are properly coordinated.

A similar expression will occur above when the x and y pixel arraysabove are replaced with their corresponding polar (spherical)coordinates as in FIG. 8 and are properly scaled.

Possibilities arise for a field of cameras where all cameras arespherical, all have their z-axes parallel, and all are scaled throughtheir radii p to match each other. In this case all images would besimilar and would be coordinated closely as spherical images.

We note that cameras which are not strictly coplanar, or fall out ofline with others, can be re-aligned with others through projectivetransformations, as may be occasionally needed.

An essential point in the present invention is that a planar structure,as in FIG. 5, appears as a practical means for coordinating and makingoperative a multitude of detectors. The transformations for new pointsof view require no rotation or projection, only affine transformations,which make for a simple and robust way to achieve high-speed performancefor recognition and tracking.

An important ramification of this invention is that with a supportingstructure as in FIG. 6 images can be constructed where none existedbefore. Cameras can be posited usefully in imaginary places and picturescan be created as though the cameras are actually there. This is shownin the dotted lines of FIG. 7 where an image is created based onparallel epipolar lines from adjacent images. In fact whole scenes canbe constructed given a handful of nearby camera images.

In FIG. 7 we posit an imaginary camera with center 13 as shown. Fromcenter 13 we run lines to adjacent camera centers—line 334 to center 8,line 344 to center 10, and line 354 to center 12. Precisely parallel tothese lines we run lines 351, 352 and 353 from the corners of image 120;we run lines 341, 342 and 343 from the corners of image 100; and we runlines 331, 332 and 333 from the corners of image 80. These linesintersect precisely on lines 211 and 212 to form the corners 131, 132and 133 of a new image 130. This is now a perspective view of object 200from a camera with center 13.

To create a 3D view of object 200 from the same distance we can positanother imaginary camera with adjacent center 14, subtend an anglesuitable for 3D at object 200, and iterate a series of lines as above.This will give us an image (not shown) precisely cognate with image 130for a full 3D perspective view of object 200.

An explanation is at hand. It lies (i) in our ability to line cameras upprecisely, as described above (the practical difference between preciseand not precise to image quality is like daylight to night) and (ii) onthe transformation M′=TM where T is an affine transformation on the x-yplane. The image 130 will be stretched along the line 211 by the ratioR′/R of the distances from the object 200 to the camera centers 13 and12. The image 130 will be compressed against line 212 by tan γ, where γis the angle subtended by the object 200 between lines 211 and 212. Thestretching and compression is automatic from the geometry. There is norotation since all the cameras will have been rotated precisely prior toadding nodes 13 and 14.

To explain further: In FIG. 5, FIG. 6 and FIG. 7, R represents how far atypical image plane 80 (containing the aggregate pixels M) is from acommon center line, a z-axis, of a group of image planes 10 . . . 120.R′ represents how far the image plane 130 (containing the aggregatepixels M′) may be from this z-axis. By changing the ratio R′/R we canmake the image 130 zoom larger or smaller, creating far and nearperspectives at will.

This can be done by a computer with an imaginary camera 130 with nophysical change to other cameras. Better yet, a pair of cameras 130 and140 can create real 3D perspectives with real images borrowed from realcameras with transformations like M′=TM above.

The images 130 and 140, as in all images, can be normalized with othersby inverting the ratio R′/R, to bring all images into conformity ofscale.

The ability of creating 3D as with nodes 13 and 14 opens up manypossibilities. For example, a whole family of 3D perspectives withimaginary cameras can be created around a few actual key viewpoints.Also, a pair of nodes such as 13 and 14 can be moved around in virtuallyreal time to obtain almost any perspective. Again, the separation ofnodes 13 and 14 can be enlarged and reduced; enlarging increases thediscrimination of objects in 3D.

Transmission speeds (as in the MPEG-4, -5 and -6 series for video) willbe increased by orders of magnitude through our ability to constructwhole scenes from a few camera images. In this scenario cameras are onlywindows: the real visual processing will be done by powerful computersat the receiving end. Pan, tilt, zoom—seeing scenes from differentperspectives—will be done remotely in virtually real-time by therecipient.

Finally, using a stabilizing program (described elsewhere by thisinventor and others, and using Newton's laws of motion) the persistenceof a scene can be continued as though actually happening for some time(i.e. seconds, or perhaps half a minute) after all cameras are blown up.Even a destroyed scene itself could be constructed to continuehypothetically for a similar period.

To a large degree certain transformations have already fixed theparameters of shape at hypothetical camera locations. What needsdiscussing are the additional parameters of color, hue, saturation,light, dark, etc. In hypothetical locations these can be inferred—forexample, green should continue as green in saturation and hue, though itmay be darkened by shadow.

The parameters above may be inferred as a weighted average from thecorresponding local images. From two adjacent images havingcorresponding pixels with different shades of green, a new color imagecould be created by summing and averaging, creating a shade in themiddle. Largely, this may be adequate.

For a projection system requiring multiple viewpoints, as exist forimmersive 3D viewing, the conversion of a handful of coordinated camerasviews, such as twelve, into multiple authentic viewpoints, such as twohundred, could be most valuable. This would be useful for both large andsmall glasses-free 3D screens.

An opportunity exists to create real reality, as opposed to augmentedreality, for popular items such as Oculus Rift and Google Cardboard.This can be done in very nearly real-time with simple transformationsusing the power of a modern cell-phone snapped into a slot behind theviewer. Football games could be watched on these devices in 3D inreal-time in detail.

In many scenarios time is limited, so the need for recognition with highprobability is critical. Multiple offset cameras with many viewpoints,as in this invention, can increase probability; the computer cancontinuously cross-correlate information from several cameras, to verifydetails; the use of image stabilization, continuity and priorscorroborates probabilities and aids identification.

The possibilities described above could be of great value to forensicwork.

To summarize: What we have shown is a method using epipolar lines andmatrix transformations to create viewpoints for which no imaging devicesexist with these following steps: (i) precisely aligning multipleimaging devices; (ii) making imaging devices coplanar by projectingimaging device epipoles to infinity; (iii) positing coplanar imaginaryimaging devices as and where needed; (iv) linking camera centers ofimaginary imaging devices to camera centers of existing imaging deviceswith baselines; (v) running epipolar lines precisely parallel tobaselines from key features of existing imaging devices to preciselyintersect at hypothetical key features of imaginary imaging devices;(vi) using matrix transformations to bring real images from imagingdevices to precisely align at hypothetical key features of imaginaryimaging devices.

What we have shown in this invention is that an efficient and exact—andtherefore fast—way of creating wholly new real images from adjacent realimages, which allows us to solve real-world imaging problems by creatingas many nearby viewpoints as necessary chosen at will.

FIG. 10 shows a field 500 of multiple coordinated detectors (“nodes”) 1,2, 3 . . . n. There can be any number of nodes in such a field. A numberof nodes (conveniently 12) can be assembled temporarily as irregularpolygons which we call “tracking skeletons”—because out of a field ofmany detectors they might represent a “bare bones” minimum for effectivetracking Examples given here are (i) a skeleton 505 consisting of nodes1, 3, 4, 5 and 7 for tracking a missile 503, which skeleton could mutaterapidly upstream to higher numbered nodes such as 11, 12, etc. dependingon the velocity of the missile; (ii) skeleton 506 consisting of nodes 6,7, 9, 10, 11 for tracking an aircraft 502; (iii) another skeleton(unnumbered) for tracking a drone 504, expecting the last skeleton tomutate slowly for a slow drone.

In FIG. 10 the volume 501 also describes an ellipsoid horopter, looselyenveloping the limits of visual perception of a system of detectors—orof our eyes. This horopter may be loosely defined as “a curve of thelimits of visual acuity”, or further defined as “a limit to the abilityof any two detectors to see an object in 3D”. We may pick an angle—say1°—for that limit, selected (arbitrarily) because smaller than that wecannot easily distinguish objects in 3D. This limit is defined as theangle subtended by an object at a particular height between two rangingdetectors. For a human being this would be 12.5 feet. Distinguishingobjects at 60 miles would take a pair of detectors separated by a mile.

The horopter 501 would also have its limits defined by the size of itsbase 500 (which is nodes), the maximum limit of 3D discrimination, andthe distance apart of its detectors for effective tracking There couldbe multiple fields like 500 for long-distance tracking, for exampletwenty-five across the width of the United States, each accounting for ahundred-mile extent.

The size of the horopter 501 would depend on the objects being trackedand their altitude. For drones 504 flying at five hundred feet thehoropter envelope could be a few hundred feet high, using detectors tenfeet apart. For aircraft 502 cruising at 60,000 feet the horopterenvelope could be twelve miles high, with detectors 1000 feet apart.Detection of both planes and drones could be combined in the same field,a rough diameter of the skeletons for each being commensurate with theirtargets' altitudes.

The extent of the horopter would also depend critically on ananticipated threat. In one scenario a missile might be approaching at600 miles an hour. Detectors a mile apart would recognize it 6 minutesaway. It might be better to allow double the minutes for response withdetectors twice the distance apart, implying a horopter twice as large.

FIG. 11 illustrates a system 700 for collecting and processing signalsfrom a field 500 of detectors 1, 2, 3, . . . n, as in FIG. 10. Whenmotion is picked up by these skyward-facing detectors, the signals arefed in parallel to a computer 702. Altitude and trajectory can beswiftly calculated, as already noted. A nearby database 701 can beconsulted for eidetic correspondences. With correspondence andtrajectory confirmed (703) a decision can be made (704) whether toaccept the object (502, 503, or 504) as harmless or to destroy it (705)with a lethal weapon (520).

As shown in Iron Dome with a similar configuration, the recentavailability of ultra-high-speed processors allows the computation ofhighly complex data in speeds approaching real-time. With fast imagerecognition algorithms and high-speed software, 3D ranging can be donein milliseconds. This allows equally fast (and automated) response toincoming missiles threatening major institutions in cities like New Yorkor Washington—all the while the missiles being unaware that they beingtracked and therefore less capable of taking evasive or jamming action.

Iron Dome uses radar, it is active. The system 500-700 is passive, withmassively parallel redundant architecture, spread over large areas withinexpensive optics, capable of using a computer the size of acell-phone, capable of multiple replication, and much harder toincapacitate.

The system portion 700 can also transmit data through the net, viasatellite or on dedicated underground fiber-optics for immediate displayor for storage.

The computer 702 in FIG. 11 processes select combinations of detectors1, 2, 3 . . . , and 10, 11, 12 . . . in small groups chosen by thecomputer in a predetermined order of significance. For example, themissile 503 may be determined the greatest threat and thereforeallocated the prime detectors and computing resources. The computedtrajectories of recognized objects such as 502, 503 and 504 (a plane, amissile and a drone) may be made available for transmission as plots andstatistics for others, even though decisions, such as shooting down themissile 503 with rockets 520 will have been predetermined by thecomputer and performed in milliseconds.

For recognition we can adapt training algorithms, such as thosedescribed by C. M. Bishop in Pattern Recognition and Machine Learning(2006). These can be simplified knowing the anticipated shape, size,color, markings etc. of the aircraft, missiles, rockets, drones, etc.expected in the area. These can be stored in the templates section 701of the computer 702. Into the algorithms will be built the expectationthat certain flying objects will recur regularly, intact and moving onappropriate trajectories. The computer 702 will also be smart enough todetect anomalies in size, speed and identification of all objects and bemade to react accordingly.

Data in an abbreviated form may be transmitted over the Internet (whichhas many redundant channels), through cellular communications channelssuch as 3G or LTE, or using Immarsat Global Xpress, all of whom providehigh-bandwidth connections. If critical the system can use undergroundfiber-optics 712 (with vast bandwidth) to remote bunkers. By whichevertransmission method the data can be decompressed and shown on remotedisplay 710, and can be sent to storage in a remote unit 711.

In more detail: In FIG. 11 the incoming data to computer 702 is taggedwith its instantaneous coordinates from a Global Positioning System.This input is fed into a processor (a section of computer 702) which hasinternal DSP functions to create the enhanced image stabilization asnecessary for images from distant objects. For transmission across theInternet another internal processor provides dual stream H.264 encoding,handles data compression, MJPEG encoding, and an output to a PhysicalLayer chip 707 for transmission over the Internet cloud 708 for remote3D viewing. The processor 702 also has an output to a wirelessconnection which uses 802.11n for 4G communication speeds. Other localchannels provided are an RS-485/RS-232 output to local storage, an HDMIoutput for 3D viewing on a local display 706.

At the receiving end a decoder 709 has the capacity to capture 500MegaPixels per second and process full 3DHD of 1080p60 for a remotedisplay 710. The rate at which scenes can unfold on this display islimited only by the vagaries of the Internet and of the wirelesschannels.

In this Codec description we are following MPEG-4, which is a collectionof methods defining compression of audio and visual (AV) digital databeginning in 1998. It was at that time designated a standard for a groupof audio and video coding formats and related technology agreed upon bythe ISO/IEC Moving Picture Experts Group (MPEG) under the formalstandard ISO/IEC 14496. In July 2008, the ATSC standards were amended toinclude H.264/MPEG-4 AVC compression and 1080p at 50, 59.94, and 60frames per second (1080p50 and 1080p60)—the last of which is used here.These frame rates require H.264/AVC High Profile Level 4.2, whilestandard HDTV frame rates only require Level 4.0. Uses of MPEG-4 includecompression of AV data for web (streaming media) and CD distributionvoice (telephone, videophone) and broadcast television applications). Wecould equally use any other protocol (or combination of protocols)suitable for transferring high-speed data over airwaves or land-lines.

This invention relates to the remote, passive ranging of objects whichare of interest to military observers and others. The addition of adimension beyond 2D replicates the vision of the human eye andcontributes the perception of depth, and adding valuable information forthe inspection of diverse (in this case flying) objects. These can beinstantaneously compared and sorted against three-dimensional (3D)templates which may reflect the ideal for that particular object. Withadvanced object recognition software inspection can be done at highspeed and with great accuracy. The images can also be compressed in realtime for high-speed transmission for remote display or analysis, or sentfor compact storage. The techniques of this invention are applicable inthe visible, infra-red, microwave and ultra-violet portions of thespectrum, and may apply also to sonar or ultrasound.

While the invention has been described and illustrated in general as amethod for recognizing, tracking and evaluating three dimensionalobjects such as aircraft and missiles, in fact to those skilled in theart, the techniques of this invention can be understood and used asmeans for creating and perfecting three-dimensional recognition,inspection and measurement tools for various subjects throughout theelectro-magnetic spectrum and beyond.

The techniques of this invention may be applied whether detectors aremoving relative to fixed objects, or objects are moving relative tofixed detectors.

It may be understood by those skilled in the art that although specificterms may be employed in this invention, they are used in a generic anddescriptive sense and must not be construed as limiting. The scope ofthis invention is set out in the appended claims.

I claim:
 1. A computer-implemented method of using a plurality ofimaging devices, positioned in a spaced apart relationship with eachother, to create a new image of an object from a new viewpoint where animaging device is not positioned, said method comprising: capturingimages of said object with each of said plurality of imaging devices;projecting epipoles from an image plane of said each of said pluralityof imaging devices to infinity; positioning all image planes coplanarwith each other; selecting a center position of a new viewpoint;linking, with baselines, said center position of said new viewpoint withcenters of plurality of imaging devices; running epipolar lines,parallel to a respective baseline, from key features of a captured imageby said plurality of imaging devices; intersecting epipolar lines fromeach respective key feature in said each of said plurality of imagingdevices to define corresponding key features in said new viewpoint; andaligning, using matrix transformations, captured images from saidplurality of imaging devices at corresponding respective key features insaid new viewpoint to create said new image.
 2. The method of claim 1,wherein said each of said plurality of imaging devices comprises acamera with a lens.
 3. The method of claim 1, wherein said plurality ofimaging devices comprises two imaging devices.
 4. The method of claim 1,further comprising tracking said object with said captured images andsaid new image, said object being a moving object.
 5. The method ofclaim 1, wherein said object comprises a plurality of objects and saidmethod further comprises tracking said plurality of objects with saidcaptured images and said new image, each of said plurality of objectsbeing a moving object.
 6. The method of claim 1, further comprisingidentifying said object with said captured images and said new image. 7.The method of claim 1, further comprising replacing an image from anyfailed imaging device from said plurality of imaging devices with saidnew image from said new viewpoint.
 8. The method of claim 1, furthercomprising calculating distances to said object from any one of saidplurality of imaging devices and said new viewpoint.
 9. The method ofclaim 8, further comprising assigning one or more of said baselinesbased on said distance(s) to said object.
 10. The method of claim 8,further comprising scaling a size of said new image in said newviewpoint.
 11. The method of claim 8, further comprising stretching andcompressing said new image based on a ratio between a distance to saidobject from said new viewpoint and a distance to said object from anyone of said plurality of imaging devices.
 12. The method of claim 8,further comprising normalizing said new image by inverting a ratiobetween a distance to said object from said new viewpoint and a distanceto said object from any one of said plurality of imaging devices. 13.The method of claim 1, wherein said positioning of said all image planescoplanar with each comprises progressively aligning said each of saidplurality of imaging devices by way of primary, secondary and finealignments.
 14. The method of claim 1, further comprising projecting acurved image field in front of a lens of said each of said plurality ofimaging devices, if a single epipole is not convergent or parallel toother epipoles.
 15. The method of claim 1, further comprising convertingpolar coordinates into orthogonal coordinates.
 16. The method of claim1, further comprising creating additional new images of said object fromadditional new viewpoints and generating a 3D image of said object. 17.A computer-implemented method for identifying a moving object with aplurality of imaging devices positioned in a spaced apart relationshipwith each other, said method comprising: capturing an image of saidmoving object with each of said plurality of imaging devices; projectingepipoles from an image plane of said each of said plurality imagingdevices to infinity; positioning all image planes coplanar with eachother; selecting a center position of at least one new viewpoint;linking, with baselines, said center position of said at least one newviewpoint with centers of plurality of imaging devices; running epipolarlines, parallel to a respective baseline, from key features of acaptured image by said each of said plurality of imaging devices;intersecting epipolar lines from each respective key feature in saideach of said plurality of imaging devices to define corresponding keyfeatures in said at least one new viewpoint; aligning, using matrixtransformations, captured images from said plurality of imaging devicesat corresponding respective key features in said at least one newviewpoint to create at least one new image of said object; and comparinga combination image comprising said at least one new image and saidcaptured image with a template image.
 18. The method of claim 17,further comprising tracking said moving object and deciding to acceptsaid object as harmless or destroy said object with a weapon.
 19. Asystem for at least identifying an object, comprising: a plurality ofimaging devices positioned in a spaced apart relationship with eachother in a polygon pattern, each capturing an image of said object; anda computer configured to: project epipoles from an image plane of saideach of said plurality imaging devices to infinity; position all imageplanes coplanar with each other; select a center position of at leastone new viewpoint; link, with baselines, said center position of said atleast one new viewpoint with centers of plurality of imaging devices;run epipolar lines, parallel to a respective baseline, from key featuresof a captured image by said each of said plurality of imaging devices;intersect epipolar lines from each respective key feature in said eachof said plurality of imaging devices to define corresponding keyfeatures in said at least one new viewpoint; align, using matrixtransformations, captured images from said plurality of imaging devicesat corresponding respective key features in said at least one newviewpoint to create at least one new image of said object; and compare acombination image comprising said at least one new image and saidcaptured images with a template image.